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Abstract 

Background: To evaluate the scientific reasoning in basic science among undergraduate medical students, we 
established the National Medical Science Olympiad in Iran. In this Olympiad, the drawing of a concept map was 
used to evaluate a student's knowledge framework; students' ability in hypothesis generation and testing were also 
evaluated in four different steps. All medical students were invited to participate in this program. Finally, 133 
undergraduate medical students with average grades > 16/20 from 45 different medical schools in Iran were 
selected. The program took the form of four exams: drawing a concept map (Exam I), hypothesis generation (Exam 
II), choosing variables based on the hypothesis (Exam III), measuring scientific thought (Exam IV). The examinees 
were asked to complete all examination items in their own time without using textbooks, websites, or personal 
consultations. Data were presented as mean ± SE of each parameter. The correlation coefficient between students' 
scores in each exam with the total final score and average grade was calculated using the Spearman test. 

Results: Out of a possible score of 200, the mean ± SE of each exam were as follows: 183.88 ± 5.590 for Exam I; 
78.68 ± 9.168 for Exam II; 92.04 ± 2.503 for exam III; 106.13 ± 2.345 for Exam IV. The correlation of each exam score 
with the total final score was calculated, and there was a significant correlation between them [p < 0.001). The 
scatter plot of the data showed a linear correlation between the score for each exam and the total final score. This 
meant that students with a higher final score were able to perform better in each exam through having drawn up 
a meaningful concept map. 

The average grade was significantly correlated with the total final score (R = 0.770), [p < 0.001). There was also a 
significant correlation between each exam score and the average grade [p < 0.001). The highest correlation was 
observed between Exam I (/? = 0.7708) and the average grade. This means students with higher average grades 
had better grades in each exam, especially in drawing the concept map. 

Conclusions: We hope that this competition will encourage medical schools to integrate theory and practice, 
analyze data, and read research articles. Our findings relate to a selected population, and our data may not be 
applicable to all medical students. Therefore, further studies are required to validate our results. 



Background 

Science is an activity that consists of the explanation, 
prediction, and control of empirical phenomena in a 
rational manner. By "scientific reasoning," we mean the 
rules of reasoning relevant to carry out this activity, 
including principles governing experimental design, 
hypothesis testing, and data interpretation. The 
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development of expertise is not only the result of 
acquiring more knowledge and skills; structuring knowl- 
edge is also a critical step. Thus, integrated frameworks 
of related concepts, which facilitate problem solving and 
other cognitive activities, are very important. Scientists 
must master two skills: recognizing where to look and 
knowing what is seen. The first ability is achieved by 
experimental design, and the second ability is hypothesis 
generation, which involves theory evaluation. The tradi- 
tional method of assessment involves measuring infor- 
mation; analysis and organization of new knowledge and 
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critical thinking with meaningful learning are often 
omitted [1-4]. 

There are many science competitions around the 
world that provide encourage students to learn about 
science. Many of them are knowledge-based while 
others focus mainly on the students presenting a 
research project and discussing their findings. 

The aim of International Olympiads, such as in phy- 
sics, chemistry, and biology, is not only to encourage 
excellence through competition, but also to enhance 
international goodwill among nations through personal 
contacts and increased understanding of the diversity of 
cultures and traditions. They also demonstrate the role 
of teamwork in solving scientific problems, stimulate 
interactions between students and teachers, and may 
help improve science education at national and interna- 
tional levels [5]. 

As an undergraduate medical student develops into a 
skilled specialist or PhD student, one important step has 
to be made, which is to bridge the gap between text- 
book knowledge and scientific skills. From textbooks, 
medical students can gather knowledge in the form of 
review articles, which represent the general consensus of 
the scientific community; however, with many research 
articles presented in journals, consensus about observa- 
tions and interpretations has generally not yet been 
reached. It is important to engage students actively in 
developing their knowledge, and it is necessary for them 
to focus on concepts applicable to solving problems and 
relating previous knowledge to new knowledge [6,7]. 

Concept mapping is a tool that can represent knowl- 
edge structure by illustrating the relationships between 
relevant concepts within a given subject domain. By 
relating and integrating new knowledge with an existing 
knowledge structure, students develop a deeper under- 
standing, allowing better use of knowledge to generate 
hypotheses, design experiments, and test the variables to 
find the answers to scientific questions [8,9]. 

The Iranian National Medical Science Olympiad was 
designed to encourage undergraduate medical students 
to bridge the gap between textbook learning and scienti- 
fic reasoning. In recent years, stem cell research has 
been grown exponentially, and stem cell-based therapies 
have the potential to improve the life of patients with 
such conditions as Parkinson disease, myocardial infarc- 
tion, and diabetes mellitus [10,11]. Therefore, stem cell 
differentiation was chosen as the subject of the present 
study. In the National Medical Science Olympiad, the 
drawing up of a concept map was used to measure 
aspects of students' knowledge framework, and their 
ability to generate and test hypotheses was also evalu- 
ated. The correlation between the students' scores in 
each step and their final total score was further 
investigated. 



Methods 

The design of the National Medical Science Olympiad in 
Iran is focused on critical thinking among medical stu- 
dents. The first Olympiad was held in Isfahan in 2009, 
and the second in Shiraz in 2010 [12,13]. The specific 
goals of the Olympiad were competition to achieve the 
following: identifying and encouraging scientifically 
talented students; generating scientific morale; helping 
students gain closer familiarity with scientific culture; 
cultural exchanges among students; encouraging team- 
work to develop creative and critical thinking; reinfor- 
cing the goals and objectives of the health system; 
further the development of interdisciplinary activities 
[12]. 

The Ethics Committee of the Olympiad approved the 
protocol adopted in the present study, and written, 
informed consent was obtained from all participants. 
Currently enrolled medical students who were interested 
in the study topic and who had an average grade of 16/ 
20 (equivalent to a GPA of about 3.2 in the United 
States or a class of about 60 in the United Kingdom) or 
higher were able to register for the study. They then 
took part in an intensive training course in the subject 
area of their choice at their own university. Enrollees 
were tested, and those with the highest grades were 
allowed to participate in the study. 

Iran has 46 medical universities, and each university is 
allowed to send only three students in the field of basic 
science to the Olympiad. A total of 133 undergraduate 
students took the test in basic science. In this study, we 
analyzed the examination results in basic science. 

Development of the Olympiad examination by the 
reference panel 

To prepare the examination used in the Olympiad, 10 
experts from different universities in Iran were chosen for 
the reference panel. These experts held PhDs in such 
areas of basic science as biochemistry, hematology, phy- 
siology, immunology, pharmacology, and pathology, 
though they had different levels of professional experience. 
Each member of the panel took each of the four tests. On 
the morning of the first day, the drawing up of a concept 
map was completed; in the afternoon, the hypothesis writ- 
ing was completed. On the morning of the second day, the 
hypothesis testing was evaluated; in the afternoon, the 
measuring of scientific thought was investigated. Each of 
the four examination periods lasted 4 h. 

Examinees 

All the medical students were invited to participate in 
this program. The selected examinees were 133 under- 
graduate medical students from 45 medical schools in 
Iran who had grades > 16/20. The length of medical 
education in Iran is 7 years. Forty-three percent of the 
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participants were male, and 57% were female. The mean 
age of the participants was 21.3 years, and their mean 
average grades were 18.3/20. The examinees were asked 
to complete all the Olympiad examination items in their 
own time without using textbooks, websites, or personal 
consultations. 

Exam I: drawing a concept map 

On the morning of the first day, the students were asked 
to draw a concept map based on three new articles in 
the field of stem cell research [14-16]. The examination 
period lasted 4 h. 

To learn how to make concept maps, each participant 
previously completed a standardized concept map train- 
ing session. Briefly, training included an introduction to 
concept mapping, followed by practice making concept 
maps on medical topics. The raters also received train- 
ing about the concept-mapping process in the National 
Medical Science Olympiad. 

Participants were given 90 min to complete their 
maps. The maps were coded, so that the identity, speci- 
alty, and level of training of the participants would be 
unknown to the raters. Participants created the maps on 
20 x 30-cm sheets of paper. In constructing their con- 
cept maps, the participants drew concepts related to a 
certain domain and then indicated links using arrowed 
lines with a proposition written above the line, describ- 
ing how the concepts were related (concept link), [8,9]. 
Map hierarchy is an important part of the process; it is 
indicated by the direction of the arrow in the concept 
link and in the arrangement of concepts in the map 
with more general concepts at the top and more specific 
concepts below. Scoring of each concept map was based 
on the following criteria, using previously published 
reports [8,9]: 

1) Valid and meaningful selection of concepts from 
the papers (score, 25%). 

2) Hierarchical arrangement of concepts with more 
general concepts at the top and more specific ones 
below (score, 20%). 

3) Meaningful integration among concepts in the 
map (score, 10%). Incorrect concept relationships 
were given zero points. A closely related concept 
was given the highest number of points. Less impor- 
tant, but correct concept relationships were given an 
intermediate score. 

4) Accuracy and depth of understanding of the rela- 
tionship (score, 20%). 

5) Degree of student creativity showing more sophis- 
ticated understanding (score, 25%). 

Propositions or cross-links that lacked linking phrases 
above the connecting line were counted separately and 



given less credit. No credit was given for an invalid (i.e., 
incorrect or wrong) proposition or cross-link. Before 
scoring the maps, the raters discussed problems with 
each other. Raters scored a random subset (40%) of the 
same maps to assess reliability. Three different raters, 
with experience in stem cell field research and blinded 
to the identity of the map author, independently scored 
each map. The raters' total scores were added to create 
a final score for each map. The use of raters who were 
familiar with both the Olympiad process and the assess- 
ment of the different steps involved in making concept 
maps was believed to be essential in obtaining low 
inter- rater variability. 

Exam II: hypothesis generation 

In the afternoon of the first day, the measuring of scien- 
tific thoughts was investigated in terms of hypothesis 
generation. After drawing their concept maps, the stu- 
dents were asked to write a hypothesis. A sample of a 
concept map that was drawn by the scientific committee 
was also available during the exam period (see Addi- 
tional file 1 and Figure 1). This exam lasted 1 h. 
The written hypothesis was evaluated by these criteria: 

1) Simplicity (score, 15%) 

2) Accuracy (score, 20%) 

3) Precision (score, 10%) 

4) Explaining the connection between events (score, 
15%) 

5) Logical coherence (score, 10%) 

6) Fruitfulness (score, 10%) 

7) Creativity (score, 20%) 

Exam III: choosing variables based on the hypothesis 

On the morning of the second day, the participants were 
asked to choose variables that were important in hypoth- 
esis testing. This step lasted 2 h. It was made clear to the 
participants that they should choose variables that helped 
prove the hypothesis they had detailed. The credit given 
for the best answer was 50%, and scientific explanation 
was scored up to an additional 50%. 

Exam IV: measuring scientific thought 

In the final step, on the afternoon of second day, the 
students were presented with a scientific finding. To 
measure the students' scientific thinking, they were 
asked a series of questions related to the finding. They 
were also asked to make suggestions for future research. 
This step also lasted 2 h. The task for the examinees 
was to evaluate their suggestions in terms of direction 
(positive, negative, or neutral) and intensity. This effect 
was captured with a Likert scale. A sample of each 
examination is given in Additional file 1. 
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Figure 1 Suggested Concept Map 



Total final score 

The total exam score was assessed using the sum of the 
four exam grades, with each exam accounting for 25% 
of the total score. 

Analysis 

The data were entered and analyzed using the Statistical 
Package for Social Sciences (SPSS) version 17.0 (SPSS, 
Inc., Chicago, IL, USA). Data were presented as mean ± 
SE of each parameter. The correlation coefficient 
between the students' scores in each exam (I to IV) with 



the total final score and average grade was determined 
using the Spearman test. A ^-value of less than 0.05 was 
considered statistically significant. 

Results 

Demographically, the study population was 43% male, 
57% female. Out of a possible score of 200, the mean ± 
SE of each exam were as follows: 183.88 ± 5.590 for 
Exam I; 78.68 ± 9.168 for Exam II; 92.04 ± 2.503 for 
Exam III; 106.13 ± 2.345 for Exam IV. The correlation 
of each exam score with the total final score was 
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calculated (Table 1), and there was a significant correla- 
tion between them (p < 0.001). This means that students 
with a better final score were able to perform better in 
each exam through having drawn a meaningful concept 
map. The scatter plot of the data showed a linear corre- 
lation between the score for each exam and the total 
final score. 

The correlation of each exam score with the medical 
university average grade was also calculated (Table 2). 
There was a significant correlation between each exam 
score and the average grade (p < 0.001). The average 
grade was also significantly correlated with the total 
final score (R = 0.770), (p < 0.001). 

The highest correlation was observed between the 
result for Exam I (R = 0.7708) and average grade. This 
means that students who had a higher average grade 
had a better grade in each exam, especially in drawing 
the concept map. 

Discussion 

Learning is not considered to be the transmission of 
data to passive receivers. It is an active process in which 
students actively develop their knowledge and map new 
information upon prior knowledge [6,7]. Therefore, 
active learning methods, such as problem-based learn- 
ing, collaborative learning, and experiential forms of 
learning, manipulate information and ideas in ways that 
help create new meanings or solve problems. A superfi- 
cial approach to learning is characterized by establishing 
a base of scientific facts through lectures and note 
taking. 

Conversely, a deep approach to learning is character- 
ized by focusing on the concepts applicable to solving 
problems and relating previous knowledge to new 
knowledge. This approach eventually results in the 
development of a collaborative learning environment 
and the use of acquired theories, concepts, and knowl- 
edge to solve new problems [6,17]. 

Heparin, insulin, the sinoatrial node, and ether 
anesthesia are just some of the discoveries that have 
been made by medical students. In the latter half of the 
nineteenth century, European medical students had to 
write and defend a research thesis to achieve their doc- 
torate in medicine. This was the platform on which the 
discoveries of Raynaud (discovery of Raynaud's disease 
or phenomenon), Langerhans (discovery of islet cells in 



Table 1 Correlations between total Olympiad 
examination score and scores in each of the four 
scientific thinking tests 



Exam IV 


Exam III 


Exam II 


Exam 1 


Correlation coefficient 


0.482 


0.487 


0.661 


0.770 


Total grade 


< 0.001 


< 0.001 


< 0.001 


< 0.001 


Significance {P value) 



Table 2 Correlations between average grade and scores 



in each of the four scientific thinking tests 



Exam IV 


Exam III 


Exam II 


Exam 1 


Correlation coefficient 


0.553 


0.557 


0.628 


0.778 


Grade point average 


< 0.001 


< 0.001 


< 0.001 


< 0.001 


Significance (P value) 



the pancreas, which secrete insulin, and also the dendri- 
tic cells in the skin known as Langerhans cells), and 
Duchesne (discovery of a type of muscular dystrophy) 
were built [18]. 

In Great Britain and North America, summer research 
projects were apparently useful for Flack (discovery of 
the sinoauricular node and heart pacemaker), McLean 
(discovery of heparin, an anticoagulant), and Best (dis- 
covery of insulin). They were the result of intense effort 
in integrating theory with practice. To quote the great 
inventor Thomas Edison (1847-1931), "Success is ten 
percent inspiration and ninety percent perspiration" 
[18]. 

In the modern world, in which different areas of 
science are rapidly developing, giving direction to medi- 
cal students is clearly very important. It is necessary to 
stress the role of basic science in medical education and 
the development of clinical reasoning skills. Thus, 
greater incorporation of basic science into the medical 
student's curriculum would seem to be required [19]. 

In this way, current knowledge about a specific topic 
derived from the literature has to be translated into 
questions and solutions obtained though the practical 
design of a project. This process requires critical think- 
ing, knowing which methods and technologies are cur- 
rently being used and developing collaborations with 
other scientists through oral and written communica- 
tion. In the present article, we describe our experience 
in a national competition to bridge the gap between 
textbooks and scientific thinking among talented medi- 
cal students. It was our aim to stimulate students to 
develop research (thinking) skills within a limited time 
frame. 

Drawing a concept map, which is based on the con- 
structivism theory of learning, was used to measure 
knowledge structure. This study demonstrates that con- 
cept map scores have a strong correlation with both 
average grades and total examination score. Better train- 
ing and more practice was associated with the ability to 
draw better maps in terms of organization of knowledge, 
understanding the relationships between concepts, and 
greater creativity. The use of concept maps by students 
as a part of their educational training has previously 
been studied. McGaghie et al. noted that following a 
focused intervention and training, students' maps relat- 
ing to concepts of pulmonary physiology became similar 
to those drawn by faculty experts [20]. 
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A high correlation between each exam result and the 
total Olympiad grade was an indicator of the concurrent 
validity of the tests in the present study and also in the 
construct validity of the whole examination. As noted 
above, women constituted 57% of the participants in 
this study. Lopatto reported that men and women did 
not differ in terms of research interest or overall plans 
to continue their education, though other authors dif- 
fered on this point [21,22]. 

The influence of the faculty mentor or supervisor on a 
student's grade and previous research experience were 
not evaluated in the present study. Russell et al. found 
little evidence of a relationship between mentor charac- 
teristics and student-reported outcomes [23]. However, 
Lopatto believed that mentoring had a significant effect 
on undergraduate research experience [24]. 

The strengths and limitations of the present study 
should be considered. The participation of a large sam- 
ple of examinees from all Iranian medical universities is 
an important strength with this study. In addition, a 
panel of experts from different medical universities 
advised the study team in the test selection and adapta- 
tion, and we think this is very important in the effort to 
achieve accurate results. 

This study was limited to a strictly screened and select 
number of the best medical students, and thus our find- 
ings may not apply to other institutions. Sampling all 
possible students who may have volunteered for this 
study at other medical schools may have produced dif- 
ferent results. Therefore, the scores cannot be generaliz- 
able to the whole population of medical students. This 
study lacked the potential for a classic experimental 
comparison with a control group. The lack of control 
groups in this type of comparison is a common problem 
[21]. 

The report of the Academic Competitiveness Council 
(U.S. Department of Education, 2007) reviewed studies 
relating to the success of Science, Technology, Engineer- 
ing, and Mathematics (STEM) education programs and 
concluded that only 10 of 115 programs were "scientifi- 
cally rigorous" by including appropriate control groups 
[25]. The creation of a proper control, such as a student 
group that applied but was not selected, is practically 
difficult. Finally, nationally and privately funded grants 
encourage institutions to recruit talented medical stu- 
dents to share in research opportunities. 

Conclusions 

The results of this study shed light onto medical student 
engagement in basic science. We hope National Medical 
Science Olympiad will improve the abilities of medical 
schools to integrate theory and practice, analyze data, 
read primary literature and research articles, develop 
skills in scientific writing, increase their confidence, and 



work independently. However, further studies are 
required to validate our assessment and to improve such 
technical aspects as computerized administration and 
test scoring. 

Additional material 



Additional file 1: Appendix 1 
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